In computer graphics, rendering is the process of producing images on a display device from descriptions of graphical objects or models. A graphics processing unit (GPU) renders 2D and 3D graphical objects, which are often represented by a combination of primitives such as points, lines, polygons, and higher order surfaces, into picture elements (pixels).
A GPU typically includes a rendering pipeline for performing rendering operations. A rendering pipeline includes the following main stages: (1) vertex processing, which processes and transforms the vertices (that describe the primitives) into a projection space, (2) rasterization, which converts each primitive into a set of pixels aligned with the pixel grid of the display with attributes such as position, color, normal and texture, (3) fragment processing, which processes each individual set of pixels, and (4) output processing, which combines the pixels of all primitives into a 2D display space.
Modern mobile devices typically offer advanced graphics capabilities. Popular mobile operating systems includes Android™, iOS™, Windows®, etc. These operating systems support a variety of graphics Application Programming Interfaces (APIs) for 2D and 3D graphics. In a mobile device, an image producer such as a GPU processes graphics commands from a graphics API to produce graphical images. A displayed image (i.e., a frame) may be the result of multiple image producers, each of which produces one or more layers of images such as the home screen, status bar, and system user interface (UI). Each image producer passes its produced image layer (also referred to as a surface) to a graphics buffer. A composing utility, such as SurfaceFlinger in Android™, is responsible for compositing the multiple image layers from the graphics buffers into a frame for the display.
In conventional graphics systems, the GPU not only produces images but also supports the operations of image composition. In some cases, the composition operations may take a significant amount of GPU resources away from its image producing tasks. For example, when launching a video game, rotating a screen, jumping back to a previous frame, or when other time-constrained and computation-intensive graphics events take place, the GPU may be called upon to perform image composition. Since the display screen has a fixed refreshing frequency (e.g., 60 Hz) marked by a periodic VSYNC signal, a heavily-loaded GPU may not be able to keep up with the refreshing speed and may cause some frames to be dropped. Therefore, there is a need to mitigate the GPU workload and to improve graphics rendering performance.